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Recognizing  that  systems  were  not  achieving  adequate  reliability  during  development,  leading 
to  reduced  warfighting  capability  and  tremendous  cost  growth,  the  Joint  Requirements 
Oversight  Council  acted  in  August  2006  to  require  a  mandatory  sustainment  key  performance 
parameter,  materiel  availability,  and  two  supporting  key  system  attributes,  materiel  reliability 
and  ownership  cost,  for  all  new  major  defense  acquisition  programs  and  other  selected  programs. 
This  article  summarizes  the  background  and  rationale  for  the  metrics  and  discusses  some  of  the 
additional  actions  being  taken  to  implement  a  disciplined  approach  to  reliability  and 
availability. 
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Whether  the  symptoms  are  pre¬ 
sented  as  quality  problems,  or 
shortcomings  in  system  reliabil¬ 
ity  as  cited  in  recent  Government 
Accountability  Office  reports,  or 
as  an  alarming  trend  in  systems  being  found  unsuitable 
during  operational  test  and  evaluation,  it  has  become 
clear  that  the  discipline  necessary  to  field  reliable,  high 
quality  systems  needs  to  be  strengthened  throughout 
the  Department  of  Defense  (DOD)  acquisition 
system.  DOD  and  Military  Service  policies  continue 
to  stress  the  importance  of  reliability,  availability, 
maintainability,  and  quality,  and  there  is  no  shortage  of 
reference  materials,  guides,  training,  and  tools  available 
to  support  the  practitioner,  yet  there  is  a  systemic 
shortfall  in  implementing  the  best  practices  necessary 
to  ensure  that  reliability  and  quality  are  cost  effectively 
engineered  into  our  systems.  Somehow,  the  engineer¬ 
ing  discipline  of  reliability  has  been  reduced  to  a  minor 
factor  associated  with  total  ownership  cost,  instead  of  a 
fundamental  characteristic  of  our  systems.  Availability 
considerations  have  faded  into  the  background  and 
many  of  our  experienced  acquisition  professionals  are 
no  longer  in  the  workforce.  Recognizing  the  need  to 
act  at  the  enterprise  level,  the  direction  taken  by  the 
Office  of  the  Secretary  of  Defense  (OSD)  Acquisition, 


Technology,  and  Logistics  (AT&L)  leadership  and  the 
Joint  Requirements  Oversight  Council  (JROC)  was 
based  on  understanding  the  integrated  acquisition  and 
sustainment  environment  as  it  exists  today,  and  was 
intended  to  use  the  primary  management  structures, 
tools,  and  processes  in  place. 

Today’s  program  managers  are  provided  very  chal¬ 
lenging  cost  and  schedule  goals  and  a  small  number  of 
key  performance  parameters  (KPPs)  and  key  system 
attributes  (KSAs)  that  must  be  met;  and  everything  else 
may  be  traded  off  if  necessary  to  meet  those  minimum 
requirements.  In  reality,  that  means  that  everything  else 
MUST  be  traded  when  necessary  to  meet  performance, 
cost,  and  schedule.  In  order  to  get  reliable,  available, 
affordable  systems,  then  rehabdity,  availabihty,  and 
ownership  cost  had  to  become  firm  requirements  instead 
of  vague  objectives.  With  the  signing  of  the  Joint 
Requirements  Oversight  Council  Memorandum  161-06 
in  August  2006,  the  JROC  established  the  requirements, 
endorsing  the  importance  of  achieving  reliability  and 
planning  for  sustainment  as  key  to  future  warfighting 
capabilities.  The  approach  taken  in  developing  the  KPP 
and  KSAs  was  to  establish  firm  requirements  for  what 
was  to  be  measured,  but  provide  maximum  flexibility  to 
the  services  and  program  sponsors  to  establish  the 
specifics  applicable  to  each  system. 
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Materiel  availability — The  capstone 
sustainment  KPP 

Materiel  Availability  is  a  measure  of  the 
percentage  of  the  total  inventory  of  a  system 
operationally  capable  (ready  for  tasking)  of 
performing  an  assigned  mission  at  a  given  time, 
based  on  materiel  condition.  This  can  he 
expressed  mathematically  as  (number  of  opera¬ 
tional  end  items/total  population). 

Materiel  Availability  also  indicates  the  percent- 
age  of  time  that  a  system  is  operationally  capable 
of  performing  an  assigned  mission  and  can  be 
expressed  as  (uptime/ (uptime  +  downtime)). 
(CJCSM  3 17  0.01  C) 

The  introduction  of  a  new  term,  Materiel  Availabil¬ 
ity,  has  naturally  raised  questions  and  concerns,  some 
of  which  will  be  addressed  by  providing  the  back¬ 
ground  and  context  for  establishing  the  new  KPP.  As  a 
metric,  materiel  availability  provides  insight  into  the 
usage  of  systems,  but  more  importantly,  requires  that 
all  of  the  major  sustainment  elements  be  considered, 
planned,  and  measured.  The  term  Materiel  Availability 
was  deliberately  chosen  to  emphasize  that  this  metric  is 
intended  to  support  the  management  of  the  acquisition 
and  materiel  readiness  processes  whose  function  is  to 
deliver  capable,  ready  systems  to  the  operational  forces, 
as  an  input  to  operational  readiness.  Operational 
readiness  is  the  concern  of  the  operational  forces  and 
must  stiU  be  measured  and  reported,  and  operational 
availability  (Aq)  no  doubt  will  remain  an  important 
metric.  Establishing  and  managing  the  materiel 
availability  metric  requires  the  consideration  of  all  of 
the  sustaining  support  that  the  acquisition  and  logistics 
professional  must  provide  to  sustain  the  capability 
being  acquired,  in  addition  to  the  reliability  and 
maintainability  characteristics  of  the  system  itself. 
The  use  of  the  term  Materiel  Availability  was  intended 
to  highlight  its  difference  from  Operational  Availabil¬ 
ity,  although  it  can  be  argued  that  the  definition  of 
materiel  availability  is  basically  A^  at  the  highest  level 
of  the  system,  across  the  entire  population.  Since  Aq  is 
widely  used  and  has  become  associated  with  very 
specific  definitions  in  each  service,  selecting  Materiel 
Availability  further  highlights  that  it  is,  by  design, 
different  from  Aq  as  implemented  across  the  services. 

The  numerical  values  for  materiel  availability  will 
often  differ  from  the  values  typically  experienced  for 
Aq,  and  may  be  significantly  lower.  The  number  itself 
provides  some  insight  into  the  planned  utility  or 
capacity  of  the  system,  but  the  number  itself  is  not  as 
important  as  the  discipline  introduced  by  the  need  to 
plan  and  manage  to  objective  values  throughout  the 


system.  Achieving  an  arbitrary  value  for  materiel 
availability  is  not  the  goal;  rather  the  numerical  value 
for  availability  should  reflect  the  actual  plans  for 
operating  and  sustaining  the  system.  For  materiel 
availability,  the  entire  population  of  the  system  must  be 
accounted  for,  as  does  all  time  during  the  planned 
service  life.  Nonoperational  units  and  time  are  included 
to  provide  a  complete  picture  of  the  investment  and 
sustainment  required  across  the  entire  program  life 
cycle. 

The  definition  of  materiel  availability  recognizes 
that  in  practical,  concrete  terms,  the  most  direct  way  to 
measure  readiness  or  availability  in  the  field  in  many 
cases  is  to  count  how  many  end  items  are  “up”  each 
day.  This  approach  works  with  some  systems,  but  not 
aU,  and  in  order  to  establish  the  metric,  analysis  of 
downtime  will  still  need  to  be  conducted. 

Generally,  while  the  capstone  metric  is  materiel 
availability,  from  the  operator’s  viewpoint  the  starting 
point  for  establishing  the  metrics  will  be  reliability, 
established  as  the  new  Materiel  Reliability  KSA.  The 
intent  of  the  KSA  is  to  establish  the  reliability 
performance  that  is  needed  to  make  the  system  useful 
in  its  intended  military  context.  This  metric  should  be 
established  with  significant  input  from  the  operational 
users  based  on  the  planned  employment  of  the  system. 
Only  the  combatant  commanders  can  really  answer  the 
question,  “How  reliable  does  the  system  need  to  be  for 
it  be  useful  in  combat?”  or  “What  probability  of  success 
must  be  achieved?” 

Materiel  Reliability  is  a  measure  of  the 
probability  that  the  system  will  perform  without 
failure  over  a  specific  interval.  Reliability  must 
be  sufficient  to  support  the  warfighting  capability 
needed.  (CJCSM  3170.01C) 

While  the  definition  for  Materiel  Reliability  in 
Chairman,  Joint  Chiefs  of  Staff  Memorandum 
(CJCSM)  3170.01C  goes  on  to  discuss  the  use  of 
mean  time  between  failure,  it  does  so  in  very  general 
terms  and  was  intended  to  allow  the  use  of  specific 
reliability  metrics  most  appropriate  to  each  system.  For 
complex,  multimission  systems,  it  may  be  appropriate 
to  establish  more  than  one  reliability  metric  or  to  use 
probability  of  mission  success  as  the  top  level  materiel 
reliability  metric.  In  general,  some  form  of  mission 
reliability  is  most  appropriate,  although  there  may  be 
cases  in  which  logistics  reliability  would  be  recom¬ 
mended  as  the  KSA.  The  selection  and  definition  of 
the  most  appropriate  metric  for  each  system  is  left  to 
the  sponsor  to  recommend  and  support.  It  is  critical  to 
define  the  operating  environments  and  mission  profiles 
in  which  the  system  is  intended  to  operate. 
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Left  unbounded  by  cost,  systems  could  achieve 
availability  objectives  by  requiring  excessive  spares, 
maintenance,  or  other  support  elements  and  reliability 
gains  could  be  pursued  beyond  the  point  of  diminish¬ 
ing  returns.  Establishing  Ownership  Cost  as  a  KSA  is 
intended  to  add  cost  discipline  beyond  that  provided  by 
the  current  approach  to  total  ownership  cost  and  life 
cycle  cost  estimates. 

Ownership  Cost  provides  balance  to  the 
sustainment  solution  by  ensuring  that  the 
operations  and  support  (O^S)  costs  associated 
with  materiel  readiness  are  considered  in  making 
decisions.  For  consistency  and  to  capitalize  on 
existing  ejforts  in  this  area,  the  Cost  Analysis 
Improvement  Group  (CAIG)  O&S  Cost  Esti¬ 
mating  Structure  will  be  used  in  support  of  this 
KSA.  Fuel  costs  will  be  based  on  the  fully 
burdened  cost  of  fuel.  Costs  are  to  be  included 
regardless  of  funding  source.  The  KSA  value 
should  cover  the  planned  lifecycle  timeframe, 
consistent  with  the  timeframe  used  in  the 
Materiel  Availability  KPP.  Sources  of  reference 
data,  cost  models,  parametric  cost  estimating 
relationships,  and  other  estimating  techniques  or 
tools  must  be  identified  in  supporting  analysis. 
Programs  must  plan  for  maintaining  the 
traceability  of  costs  incurred  to  estimates  and 
must  plan  for  testing  and  evaluation.  The 
planned  approach  to  monitoring,  collecting,  and 
validating  operating  and  support  cost  data  to 
supporting  the  KSA  must  be  provided.  (CJCSM 
3 17 0.01  C) 

Since  acquisition  costs  are  intensely  monitored 
already,  the  KSA  is  focused  on  O&S  costs,  and  is 
intended  to  elevate  management  attention  to  06cS  cost 
considerations.  However,  it  was  also  recognized  that  the 
quality  and  completeness  of  06cS  cost  data  available  is 
less  than  that  of  acquisition  cost  data,  and  the 
connection  between  the  O&S  estimates  and  the 
eventual  costs  incurred  is  soft  in  some  areas.  Over  the 
long  term,  this  area  wiU  continue  to  mature,  with  the 
objective  of  eventually  being  able  to  rely  on  the  O&S 
cost  KSA  values  as  the  basis  for  planning  and  budgeting. 

Only  the  cost  elements  most  directly  associated  with 
materiel  readiness  are  required,  but  program  sponsors 
are  free  to  add  other  cost  elements  if  appropriate.  For 
example,  manpower  costs  are  not  required,  but  there 
are  some  systems  for  which  manpower  costs  are  the 
focus  of  significant  program  effort  and  should  be 
included.  The  CAIG  O&S  Cost  Estimating  Structure 
is  used  so  that  the  KSA  does  not  create  a  new  or 
different  cost  structure  that  would  differ  from  that  used 
for  other  program  estimates. 


Supporting  analysis 

Clearly  ah  of  the  analysis  required  to  establish  the 
KPP  and  KSAs  wih  not  be  included  in  the  Capability 
Development  Document  (CDD)  or  Capability  Pro¬ 
duction  Document  (CPD)  with  the  established  KPP 
and  KSAs.  To  support  the  immediate  requirements  of 
programs  submitting  CDDs  or  CPDs  for  JROC 
approval,  the  Office  of  the  Deputy  Under  Secretary 
of  Defense  for  Logistics  and  Materiel  Readiness  and 
the  Joint  Staff  J4  developed  a  “Guide  to  the 
Sustainment  KPP”  issued  by  J4  and  available  on  the 
Joint  Staff  Knowledge  Management  and  Decision 
Support  system.  The  Guide  defines  the  requirements 
for  supporting  analysis  and  the  process  by  which 
recommendations  for  KPPs  and  KSAs  are  reviewed. 
While  the  Guide  does  not  mandate  specific  formats  or 
products,  it  does  describe  the  criteria  by  which  the 
analysis  will  be  evaluated.  A  more  definitive  document 
is  currently  in  development  which  wih  formalize  and 
standardize  the  required  analytical  products.  Currently 
titled  the  “DoD  Reliability,  Availability,  Maintain¬ 
ability  and  Cost  Rationale  Report  Handbook,”  it  is 
intended  to  support  three  primary  objectives: 

1.  Provide  guidance  in  developing  and  documenting 
realistic  sustainment  KPP/KSA  requirements  and 
the  related  supporting  rationale; 

2.  Provide  guidance  to  the  acquisition  community 
to  understand  how  the  requirements  must  be 
measured  and  tested  throughout  the  life  cycle; 

3.  Describe  the  processes  for  OSD  AT&L,  the 
Joint  Staff  and  other  stakeholders  to  follow  in 
interfacing  with  the  Services  and  programs  in 
developing  sustainment  requirements. 

Using  the  processes  established  in  the  handbook  wih 
assist  in  assessing  alternatives  considered  during  the 
Analysis  of  Alternatives,  and  in  articulating  the 
requirements  and  the  supporting  rationale  needed  for 
the  CDD  and  CPD.  Subject  matter  experts  from  OSD 
AT&L,  DOT&E  and  the  Joint  Staff  have  worked 
together  to  develop  the  handbook  which  was  inspired 
by  the  “RAM  Rationale  Report”  used  in  the  past  by  the 
Army. 

Test  implications 

The  test  community,  particularly  the  OSD 
DOT&E  leadership,  has  actively  campaigned  for 
renewed  attention  to  setting  and  achieving  perfor¬ 
mance  goals  during  the  product  development  process 
well  in  advance  of  the  operational  evaluation.  It  is 
certainly  preferable  for  the  user  community  to  define 
the  required  availability  and  reliability  that  would 
constitute  a  useful  (suitable)  system  than  it  is  to  leave 
that  determination  entirely  subjective  on  the  part  of  the 
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evaluator.  A  concern  frequently  voiced  is  that  proba¬ 
bilistic  measures  such  as  availability  and  reliability  are 
difficult  to  demonstrate  during  the  operational  test 
timeframe,  and  cost  is  almost  impossible  to  verify 
directly.  If  we  continued  to  view  operational  testing  as 
a  one  time,  pass/fail  event  as  in  the  past,  these  concerns 
would  be  significant.  Given  the  renewed  leadership 
emphasis  on  using  an  integrated  approach  to  testing 
intended  to  build  confidence  throughout  the  entire 
process  leading  to  the  operational  test  event,  it 
certainly  should  be  feasible  to  incrementally  build  up 
confidence  in  all  of  the  sustainment  elements. 

Future  steps 

Within  OSD,  the  new  metrics  have  been  endorsed 
by  the  Deputy  Under  Secretary  of  Defense  for 
Logistics  and  Materiel  Readiness  for  use  even  where 
they  are  not  mandated  as  KPP  or  KSA  values.  Defense 
Acquisition  Executive  Summary  reporting  now  in¬ 
cludes  sustainment,  based  on  the  new  metrics  dis¬ 
cussed.  Gradually,  programs  are  developing  bench¬ 
marks  and  assessing  their  status  and  contributing  to 
our  collective  experience.  Policy  and  guidance  docu¬ 
ments  will  address  the  metrics  in  their  next  revisions, 
and  the  status  of  these  metrics  is  being  added  to  major 
reviews.  While  the  establishment  of  the  mandatory 
Sustainment  KPP  is  the  cornerstone  of  our  efforts, 
there  is  certainly  additional  work  to  be  done  to  improve 
our  ability  to  build  in  reliability  and  sustainment  up 
front  during  the  development  stages  of  programs. 
Results  from  initial  pilot  programs  exploring  the 
relationship  between  funding  and  materiel  readiness 
must  be  reviewed  and  action  taken.  There  are 
improvements  needed  in  collecting  and  analyzing 
system  performance  data  across  the  enterprise,  as  well 
as  in  improving  our  OScS  cost  data  collection  and 
analysis.  Shortfalls  in  our  workforce  skills  are  being 


identified  and  solutions  are  being  developed.  Finally, 
realistic  approaches  to  testing  the  sustainment  metrics 
will  need  to  be  established. 

There  is  nothing  more  basic  in  the  development  of  a 
weapon  system  than  ensuring  that  when  it  is  employed 
in  combat,  it  will  work  when  someone’s  life  hangs  in 
the  balance.  The  progress  made  in  restoring  this 
discipline  to  the  development  process  is  significant, 
and  will  pay  off  in  enhanced  readiness  and  reduced  cost 
over  the  life  cycle  of  our  weapon  systems. 
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